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Response to Amendment and Arguments 

1 . Applicant's amendment filed March 1 8, 2009, has been entered and made of 
record. 

2. The rejection of claims 1-3 and 5-8 under 35 U.S.C. 101 is withdrawn because the 
Applicant amended to tied to a particular machine "video phone system" and "image 
processor" to the method claims to perform the aforementioned limitations. 

3. The rejection of claims 9-11, 13-19 and 21-23 under 35 U.S.C. 1 12, second paragraph 
is withdrawn because the Applicant's arguments and amendment considered persuasive. 

4. Applicant's arguments with respect to claims 1-3, 5-11, 13-19 and 21-23 have 
been considered but are moot in view of the new ground(s) of rejection. Please see the 
rejections as indicated below. 

The Examiner believes that all the arguments of the Applicant have been properly 
addressed and explained. Thus, the rejections of all of the claims are maintained. 

Examiner's Note 

1 . Examiner has cited particular columns and line numbers or figures in the references as 
applied to the claims below for the convenience of the applicant. Although the specified citations 
are representative of the teachings in the art and are applied to the specific limitations within the 
individual claim, other passages and figures may apply as well. It is respectfully requested from 
the applicant, in preparing the responses, to fully consider the references in entirety as potentially 
teaching all or part of the claimed invention, as well as the context of the passage as taught by 
the prior art or disclosed by the examiner. 
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Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 1-3, 5-11, 13-19 and 21-23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of U.S. Patent 6,806,898 by Toyama et al. ("Toyama"), U.S. 
Patent 6,707,933 by Mariani et al. ("Mariani"), Schneiderman et al. U.S. Pub. No. 
2002/0159627, and further in view of Kim U.S. Pub. No. 2003/0064658. 

Regarding claim 1, Toyama discloses a method (figure 8) for processing an image 
containing at least a portion of a head of a human in a video phone system, comprising: 

estimating an orientation of said head in said image using a pattern recognition technique 

(810); 

computing a three dimensional model of a face surface of said human using a computer 
vision technique (822, 824); and 

adjusting an orientation of said three dimensional face surface model to provide a frontal 
view (826). 

Toyama does not explicitly disclose a pattern recognition technique comprises a 
classification technique and does not appear to disclose that the image is kept unmodified when 
the orientation of the head is estimated to be frontal, as claimed. It appears that Toyama 
performs the computing and adjusting steps regardless of the orientation of the head. Also, 
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Toyama does not explicitly teach computing a three dimensional model on a face surface of said 
human using a computer vision technique based on the result of the classification technique. 

Mariani discloses a facial direction estimation system. In particular, for a video 
conferencing environment, Mariani teaches that "[u]sing the face direction estimation ... it is 
possible to enhance the quality of the transmitted images by generating a frontal view when the 
face is off-frontal ... or by unchanging the current satisfying frontal face which is displayed" 
(column 3/34-40) and further teaches a pattern recognition technique (column 3, lines 63-65) 
comprises a classification technique (column 4, lines 25-28). Thus, Mariani, like Toyama, 
recognizes the need to adjust the orientation of the face in order to achieve eye-to-eye contact for 
video conferencing communications. Mariani further recognizes that when the original image of 
the face is a "satisfying frontal face" there is no need to change the orientation of the head, and 
the image can be left unchanged. Accordingly, it would have been obvious to one of ordinary 
skill in the art at the time of the invention to modify Toyama by Mariani to achieve the claimed 
invention since Mariani teaches that when the orientation of the head is estimated to be front- 
facing, there is no need to modify the image of the face since it is already in the desired 
orientation. 

Additionally, Schneiderman teaches a facial processing system (FIG. 6) wherein 
discloses computing a three dimensional model on a face surface of said human using a 
computer vision technique based on the result of the classification technique (a processing of 
three dimensional model utilizing decision rule mathematically to train classifiers) (page 4, 
[0054] and page 5, column 2). Modifying Toyama's method of processing human facial 
according to Schneiderman would be able to compute a three dimensional model on a face 
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surface of said human using a computer vision technique based on the result of the classification 
technique. This would improve processing because it would improve estimation of the presence 
of the 3D object based on evaluation of the plurality of visual attributes (page 2, [0014]) and 
therefore, it would have been obvious to one of the ordinary skill in the art to modify Toyama 
according to Schneiderman. 

Toyama does not explicitly teach a concept of the camera and the display of the handheld 
video phone system are integrated in a single unit and thus would not be able to provide an 
image containing at least a portion of a head of a user of the video phone system to the image 
processor. Kim teaches a handheld video phone system (abstract) wherein further discloses a 
concept of the camera and the display of the handheld video phone system are integrated in a 
single unit (abstract; FIG. 1, 4-5) and thus would be able to provide an image containing at least 
a portion of a head of a user of the video phone system to the image processor. Modifying 
Toyama 's method of processing human facial according to Kim would be able to teach a concept 
of the camera and the display of the handheld video phone system arc integrated in a single unit 
and thus improve the portability. This would improve processing and therefore, it would have 
been obvious to one of the ordinary skill in the art to modify Toyama according to Kim. 

Regarding claim 2, Toyama discloses said computing act further comprises an act of 
using a symmetric face assumption to obtain a complete three dimensional face surface model 
for a profile view (824). 

Regarding claim 3, Toyama discloses said computing act further comprises an act of 
employing a structure from motion technique to obtain said three dimensional face surface model 
(column 13/1-10). 
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Regarding claim 5, Toyama discloses said computing step generates a morphable three 
dimensional model (column 10/60 — 1 1/43: the 3D model of the face can be morphed, i.e., 
rotated, moved, changed, etc., into a desired shape/pose). 

Regarding claim 6, Toyama discloses the step of mapping said three dimensional face 
surface model having an adjusted orientation to a two dimensional space (828). 

Regarding claim 7, Toyama discloses the step of transmitting said adjusted image to a 
remote user (see e.g. figure 2). 

Regarding claim 8, Toyama discloses the step of presenting said adjusted image to a local 
user (see e.g. figures 1 and 2). 

Regarding claim 9, Toyama discloses an image processor (102, figure 1) for use in a 
video phone system, comprising: 

a memory (104) for storing an image containing at least a portion of a head of a human; 

and 

a head pose corrector that 

(i) estimates an orientation of said head in said image using a pattern recognition 
technique (810, figure 8); 

(iii) computes a three dimensional model of a face surface of said human using a 
computer vision technique (822, 824, figure 8); and 

(iv) adjusts an orientation of said three dimensional face surface model to provide 
a frontal view (826, figure 8). 
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Toyama does not appear to disclose that the image is kept unmodified when the 
orientation of the head is estimated to be frontal, as claimed. It appears that Toyama performs the 
computing and adjusting steps regardless of the orientation of the head. 

Mariani discloses a facial direction estimation system. In particular, for a video 
conferencing environment, Mariani teaches that "[u]sing the face direction estimation ... it is 
possible to enhance the quality of the transmitted images by generating a frontal view when the 
face is off-frontal ... or by unchanging the current satisfying frontal face which is displayed" 
(column 3/34-40). Thus, Mariani, like Toyama, recognizes the need to adjust the orientation of 
the face in order to achieve eye-to-eye contact for video conferencing communications. Mariani 
further recognizes that when the original image of the face is a "satisfying frontal face" there is 
no need to change the orientation of the head, and the image can be left unchanged. Accordingly, 
it would have been obvious to one of ordinary skill in the art at the time of the invention to 
modify Toyama by Mariani to achieve the claimed invention since Mariani teaches that when the 
orientation of the head is estimated to be front-facing, there is no need to modify the image of the 
face since it is already in the desired orientation. 

Toyama does not explicitly teach a concept of the camera and the display of the handheld 
video phone system are integrated in a single unit and thus would not be able to provide an 
image containing at least a portion of a head of a user of the video phone system to the image 
processor. Kim teaches a handheld video phone system (abstract) wherein further discloses a 
concept of the camera and the display of the handheld video phone system are integrated in a 
single unit (abstract; FIG. 1, 4-5) and thus would be able to provide an image containing at least 
a portion of a head of a user of the video phone system to the image processor. Modifying 
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Toyama's method of processing human facial according to Kim would be able to teach a concept 
of the camera and the display of the handheld video phone system are integrated in a single unit 
and thus improve the portability. This would improve processing and therefore, it would have 
been obvious to one of the ordinary skill in the art to modify Toyama according to Kim. 

Regarding claim 10, Toyama discloses said head pose corrector is further configured to 
use a symmetric face assumption to obtain a complete three dimensional face surface model for a 
profile view (824, figure 8). 

Regarding claim 11, Toyama discloses said head pose corrector is further configured to 
employ a structure from motion technique to obtain said three dimensional face surface model 
(column 13/1-10). 

Regarding claim 13, Toyama discloses said three dimensional face surface model is a 
morphable three dimensional model (column 10/60 — 1 1/43: the 3D model of the face can be 
morphed, i.e., rotated, moved, changed, etc., into a desired shape/pose). 

Regarding claim 14, Toyama discloses said head pose corrector is further configured to 
map said three dimensional face surface model having an adjusted orientation to a two 
dimensional modified image (828, figure 8). 

Regarding claim 15, Toyama discloses said two dimensional modified image is 
transmitted to a remote user (see e.g. figure 2). 

Regarding claim 16, Toyama discloses said two dimensional modified image is presented 
to a local user (see e.g. figures 1 and 2). 

Regarding claim 17, Toyama discloses a video phone system (figures 1-2), comprising: 
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a memory (104, figure 1) for storing an image containing at least a portion of a head of a 
human; and 

a head pose corrector (102, figure 1) that 

(i) estimates an orientation of said head in said image using a pattern recognition 
technique (810, figure 8); 

(iii) computes a three dimensional model of a face surface of said human using a 
computer vision technique (822, 824, figure 8); and 

(iv) adjusts an orientation of said three dimensional face surface model to provide 
a frontal view (826, figure 8). 

Toyama does not appear to disclose that the image is kept unmodified when the 
orientation of the head is estimated to be frontal, as claimed. It appears that Toyama performs the 
computing and adjusting steps regardless of the orientation of the head. 

Mariani discloses a facial direction estimation system. In particular, for a video 
conferencing environment, Mariani teaches that "[u]sing the face direction estimation ... it is 
possible to enhance the quality of the transmitted images by generating a frontal view when the 
face is off-frontal ... or by unchanging the current satisfying frontal face which is displayed" 
(column 3/34-40). Thus, Mariani, like Toyama, recognizes the need to adjust the orientation of 
the face in order to achieve eye-to-eye contact for video conferencing communications. Mariani 
further recognizes that when the original image of the face is a "satisfying frontal face" there is 
no need to change the orientation of the head, and the image can be left unchanged. Accordingly, 
it would have been obvious to one of ordinary skill in the art at the time of the invention to 
modify Toyama by Mariani to achieve the claimed invention since Mariani teaches that when the 
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orientation of the head is estimated to be front-facing, there is no need to modify the image of the 
face since it is already in the desired orientation. 

Toyama does not explicitly teach a concept of the camera and the display of the handheld 
video phone system are integrated in a single unit and thus would not be able to provide an 
image containing at least a portion of a head of a user of the video phone system to the image 
processor. Kim teaches a handheld video phone system (abstract) wherein further discloses a 
concept of the camera and the display of the handheld video phone system are integrated in a 
single unit (abstract; FIG. 1, 4-5) and thus would be able to provide an image containing at least 
a portion of a head of a user of the video phone system to the image processor. Modifying 
Toyama 's method of processing human facial according to Kim would be able to teach a concept 
of the camera and the display of the handheld video phone system are integrated in a single unit 
and thus improve the portability. This would improve processing and therefore, it would have 
been obvious to one of the ordinary skill in the art to modify Toyama according to Kim. 

Regarding claim 18, Toyama discloses said head pose corrector is further configured to 
use a symmetric face assumption to obtain a complete three dimensional face surface model for a 
profile view (824, figure 8). 

Regarding claim 19, Toyama discloses said head pose corrector is further configured to 
employ a structure from motion technique to obtain said three dimensional face surface model 
(column 13/1-10). 
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Regarding claim 21, Toyama discloses said head pose corrector is further configured to 
map said three dimensional face surface model having an adjusted orientation to a two 
dimensional modified image (828, figure 8). 

Regarding claim 22, Toyama discloses said two dimensional modified image is 
transmitted to a remote user (see e.g. figure 2). 

Regarding claim 23, Toyama discloses said two dimensional modified image is presented 
to a local user (see e.g. figures 1 and 2). 

Conclusion 

1 . Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of the 
advisory action. In no event, however, will the statutory period for reply expire later than 
SIX MONTHS from the date of this final action. 
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Contact Information 

1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to BRIAN Q. LE whose telephone number is (571)272-7424. The 
examiner can normally be reached on 8:30 A.M - 5:30 P.M. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Samir Ahmed can be reached on 571-272-7413. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Brian Q Le/ 

Primary Examiner, Art Unit 2624 
Friday, May 15, 2009 



